A kernel based adaline

By expanding a function in series form it can be represented to an arbitrary degree of accuracy by taking enough terms. It is therefore possible, in principle, to conduct a linear regression on a new set of variables, transformed by a fixed mapping. This leads to a large computational burden and to the need for an infeasible amount of data from which the coefficients must be estimated and is not generally practical for function approximation. The algorithm studied in [1] is a linear Perceptron, which is computed implicitly in a space of infinite-dimension (the linearisation space) using potential (kernel) functions. These have been further exploited in the Support Vector Machine (SVM) [2], principal component analysis [3], linear programming machines [4] and clustering [5]. The kernel-Adatron [4,6,7], provides a fast, simple, and robust alternative to SVM classifiers providing arbitrary, large-margin discriminant functions iteratively, so avoiding the intensive QP computations of the SVM. We now use kernels to develop a non-linear version of the Adaline [8], yielding a general, non-linear adaptive mapping device via an algorithm with well-documented properties. Selecting an appropriate kernel and its parameter specifies the mapping to the linearisation space, which can be done empirically, via cross validation.